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ABSTRACT 

The Arabic version of I. G. Saras on's (1984) 

Reactions to Tests scale was used to examine the impact of using item 
parcels on ad hoc goodness of fit indices in confirmatory factor 
analysis. Item parcels with different numbers of items and different 
numbers of parcels per factor were used. Analyses were conducted on a 
sample of 420 tenth graders from 2 similar Israeli-Arab high schools. 
Model fit was examined in terms of chi square, chi square to degrees 
of freedom ratio, the goodness“of“f i t index (GFl) , the comparative 
fit index (CFl) , the Tucker-Lewis index (TLl) , and the single sample 
expected cross-validation index (ECVI) . Model fit in terms of GFI, 
CFI, TLI , and ECVI systematically improved when the number of items 
per parcel increased. The P value associated with chi square 
indicated the same tendency. However, the chi square to degrees of 
freedom ratio did not change systematically. The analyses were 
repeated with another sample consisting of 374 tenth graders from the 
same 2 high schools. Basically, the results were the same as those of 
the first sample. (Contains 2 figures, 6 tables, and 20 references.) 
(Author/SLD) 
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Abstract 

The Arabic version of Sarason's Reactions to Tests (RTT) 
scale was used to examine the impact of using item parcels on ad 
hoc goodness of fit indices in confirmatory factor analysis. Item 
parcels with different numbers of items and different numbers of 
parcels per factor were used. Analyses were conducted on a sample 
of 420 tenth graders from two similar Israeli-Arab high schools. 
Model fit was examined in terms of chi-square, chi-square to 
degrees of freedom ratio, the goodness-of -f it index (GFI), the 
comparative fit index (CFI), the Tucker-Lewis index (TLI), and the 
single sample expected cross-validation index (ECVI) . Model fit in 
terms of GFI, CFI, TLI, and ECVI systematically improved when the 
number of items per parcel increased. The P value associated with 
chi-square indicated the same tendency. However, the chi-square to 
degrees of freedom ratio did not change systematically. The 
analyses were repeated with another sample consisting of 374 tenth 
graders from the same two high schools. Basically, the results 
were the same as those of the first sample. 
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Introduction 

Item Parcels 

Most questionnaires used in research on personality 
constructs consist of ordered categorical items. Usually, the 
method of maximum likelihood (ML) is used in factor analytic 
studies involving such items. This estimation method is based on 
the assumption that the data are continuous and normally 
distributed. These assumptions are frequently violated, especially 
when categorical items are analyzed, and the violation can result 
in misleading findings and conclusions about the factor structure 
(Bernstein & Teng, 1989; Kishton & Widaman, 1994) . 

One solution to this problem is to use item parcels. An item 
parcel is a simple sum of several items assessing the same 
construct (Kishton & Widaman, 1994; see Figure 1 for an example of 
parceling) . This concept is also referred to as testlet (Wainer, & 
Kiely, 1987) or miniscale (Prats, 1990). By creating item parcels, 
new variables are constructed that are closer to continuous 
variables and allow for a distribution closer to normal. Thus, 
item parcels are more likely to meet the assijmptions of maximum 
likelihood estimation than individual ordered-categorical items. 

In other words, parceling can be viewed as a heuristic approach to 
converting ordered-categorical data into more continuous data with 
an eye toward minimizing the size of the attenuation caused by 
using ordered-categorical variables. Thus, the primary reason for 
developing item parcels is to yield variance-covariance matrices 
that are amenable to a linear factor analysis (Schau, Stevens, 
Dauphinee, & Vecchio, 1995) . 
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Formation of item parcels . Item parcels have been used in 
both personality and aptitude tests. Researchers have applied 
different methods to sum subsets of items to form item parcels. In 
the context of aptitude tests, item parcels have been constructed 
as one way of dealing with a variety of problems that might occur 
in the algorithmic construction of tests. Principal among these 
difficulties are problems with context effects, item-ordering, and 
content balancing (Wainer & Lewis, 1987) . Item parcels have also 
been used for testing dimensionality of aptitude tests consisted 
of dichotomous items. For example, item parcels were created in a 
way such that they have approximately the same means and standard 
deviations. The parcels were formed from non-parallel items, which 
have different levels of difficulty and non -over lapping content 
(Cook, Dorans, & Eignor, 1988) 

In the context of personality measures, item parcels have 
been used to improve psychometric properties of measures and to 
have a smaller number of observed variables. Item parcels have 
been constructed in several ways. Cattell and Burdsal (1975) 
primarily used the factor analysis method to form item parcels. To 
compare the performance of different methods of parceling in terms 
of model fit in confirmatory factor analysis (CFA) , Kishton and 
Widaman (1994) compared unidimensional and domain representative 
parcels and reconimended the latter method for improving the 
psychometric properties of behavioral measures of personality. 

To our knowledge, only in one study, individual items were 
contrasted with item parcels in terms of their impact on overall 
model fit in CFA context. In that study, Prats (1990) used data 
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from the Test Anxiety Inventory to construct item parcels 
consisted of random combinations of two items. Her results were 
inconsistent in terms of model fit. That is, item parcels did not 
always perform better than individual items in terms of model fit . 

Number of items per parcel . The effect of the number of items 
per parcel on the model fit has not been thoroughly investigated. 

In previous studies, the number of items per parcel in factor 
analytic methods were used varies from two (Prats, 1990) to 
eighteen (Kishton & Widaman, 1994) . However, the impact of using 
different numbers of items per parcel has not been explored. 
Evaluation of Model Fit 

The impact of using parcels has been evaluated in terms of 
reliability, factor loadings, and model fit (Cool< et al . 1988; 
Kishton & Widaman, 1994; Prats, 1990). Model fit in confirmatory 
factor analysis (CFA) is assessed by a variety of indices 
developed by several researchers ( for a recent review of indices 
of model fit, see Hu & Bentler, 1995). Indices of tit are 
classified into two broad categories: absolute indices of fit 
which include chi-square, the Goodness of Fit Index (GFI) , and the 
Expected Cross-Validation Index (ECVI) ; and three types of 
incremental indices of fit such as the Bentler and Bonnet's Normed 
Fit Index (NFI, I/pe I), the Tuclcer Lewis Index (TLI, Type II), 
and the Comparative Fit Index (CFI, Type III) . There is no 
consensus among researchers with regard to the best combination of 
indices to evaluate model fit. However, Hoyle and Panter (1995) 
recommend using the absolute indices of fit in conjunction with a 
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representative of both the type II and III incremental indices to 
evaluate model fit. 

The chi-souare test . The conventional overall test of fit in 
covariance structure analysis assesses the magnitude of the 
discrepancy between the observed covariance matrix and the 
covariance matrix implied by the model. Under an assumed 
distribution and the hypothesized model E(0) for the population 
covariance E, the test statistic T = U^~^) F^in has an asymptotic 

(large sample) chi-square distribution. The statistic T is often 
denoted "the chi-square test." 

In general, the Hq : E = Z(0) is rejected if the value of the T 
statistic exceeds a based upon chi-square distribution at a 

given alpha level of significance. However, in the context of 
covariance structure analysis, the chi-square test is used as a 
descriptive device rather than a statistical test of fit. A chi- 
square value to its degrees of freedom ratio less than two 
suggests a reasonable model fit (Carmines, & Mclver, 1981) . 

Because of the problems associated with chi-square test such 
as the sample size, statistical power, and violation of the 
multivariate normality assumption, the standard chi-square test 
may not be a sufficient guide to model fit. Also a chi-square test 
offers only a dichotomous decision strategy implied by a 
statistical decision rule and cannot be used to quantify the 
degree of fit along a continuum with some prespecified boundary. 
Thus, many so-called fit indices have been developed to assess the 
degree of congruence between the model and data (Bentler, 1990, 
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Bentler & Bonnet, 1980; Joreskog & Sorbom, 1981 Tucker & Lewis, 
1973) . 

Goodness of Fit Index (GFI) . Joreskog and Sorbom (1984) 
proposed GFI as a measure of the relative amount of variance and 
covariance in observed data that are accounted for by the 
covariance matrix of implied model . GFI carries an intuitive 
interpretation because it is analogous to the familiar value 
often reported in the context of multiple regression (Tanaka, 

1993) . 

Tucker-Lewis Index (TLI) . TLI is the classic index first 
developed by Tucker and Lewis (1973) under the assumption of 
normality and for the use of the ML estimation method uses 
information from the expected value of the chi-square of the 
target model. It was developed to quantify the degree to which a 
particular factor model is an improvement over a zero factor model 
when assessed by maximum likelihood. TLI is interpreted as a 
relative decrease in noncentrality per degrees of freedom. Thus, 
the TLI has an embodied parsimony component. Additionally, this 
index is not bounded by zero and one. 

Comparative Fit Index (CFI) . Similar to the TLI, the CFI 

(Bentler, 1990) can be interpreted as a comparative reduction in 
noncentrality with respect to a null model. According to Gerbing 
and Anderson (1993), CFI provides a more precise measure of fit 
than TLI because of its smaller empirical standard error. CFI is 
bounded lay zero and one. Therefore, it is conceptually easier to 
interpret the CFI than to interpret the TLI . 
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Expected cross-validation Index (ECVI) . The expected cross- 
validation index (ECVI) for a single sample as proposed by Browne 
and Cudeck (1989, 1993) reflects the expected discrepancy over all 
possible calibration samples. Smaller values of the ECVI indicate 
a higher probability that the model will be replicable across 
'samples from the same population. For example, when the 90% 
confidence interval corresponding to ECVI includes zero, the model 
fits the data from future samples drawn from the same population 
90% of the time. 

TLI and CFI were chosen because they seem to be less 
influenced by sample size than other fit indices are (Hu & 

Rentier, 1995; Hoyle & Panter, 1995) . Judging from the literature, 
the acceptable evaluation criteria for the listed indices are as 
follows: to indicate reasonable fit, chi-square to degrees of 
freedom- ratio should not exceed 2.00, and the values of GFI, TLI, 
and CFI should exceed .90. For ECVI smaller value indicates better 
model fit. 

Purposes of the Study 

Although item parcels have been used and some guidelines have 
been developed to aid researchers forming item parcels properly, 
to our knowledge, no studies in the literature investigated 
performance of different numbers of items per parcel. Therefore, 
the purposes of this study are to investigate (a) the impact of 
using item parcels vs. individual items and (b) the impact of the 
number of items per parcel on model fit as assessed by a variety 
of overall absolute and incremental fit indices. 
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Method 

Subjects 

In the present study, two samples were used. The first sample 
consists of 420 tenth graders aged 15 to 16 from two Arab high 
schools in the central district of Israel. Of the participants, 

194 were male and 226 were female students. The second 
(replication) sample includes 374 (147 male and 227 female) 
students w'.^h similar characteristics to the students in the first 
sample. 

Instrument 

The Reaction to Test (RTT) scale developed by Sarason (1984) 
consists of 40 Likert-type items with a four-point response scale. 
The RTT scale used in this study was translated into Arabic by the 
second author. The theoretical structure of the RTT scale consists 
of four subscales referred to as 'worry, ' 'tension, ' 'test 
irrelevant thinking, ' and 'bodily symptoms.' Each of the four 
original subscales includes ten items (Sarason, 1984) . The four- 
factor model was used to compare models which consist of different 
numbers of parcels formed from different numbers of items. 

Data Analysis 

To compare the influence of different sizes and numbers of 
item parcels, four different item sets were analyzed. First, all 
40 items were used to form two parcels consisted of five items for 
each factor and five parcels consisted of two items for each 
factor. Second, 36 items were selected based on content, item 
intercorrelations, and item reliability. These items were used to 
form three parcels per factor, each is consisted of three items. 
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Third, 32 items were selected based on the same criteria, and two 
and four parcels per factor were formed. Finally, 24 items were 
selected in the same manner to form two and three parcels per 
factor (see Figure 2) . 

Model fit was evaluated in terms of indices of fit obtained 
by the CALIS procedure in SAS 6.04. These indices include chi- 
square and its associated p value, chi-square to degrees of 
freedom ratio, GFI, TLI, CFI , and ECVI . All the analyses described 
above were repeated with a replication sample to test the 
stability of the results. 

Results 

First Sample 

The results are summarized in terms of descriptive 
statistics, reliability (squared multiple correlation: SMC), and 
model fit across models using individual items vs. different sizes 
and different numbers of item parcels. 

Descriptive statistics and the reliability of the individual 
items are shown in Table 1, and a sample of those corresponding to 
item parcels are shown in Table 2. Descriptive statistics of each 
model shows that the univariate normality of each indicator in 
terms of skewneiss and kurtosis is ii.ijroved when item parcels are 
used. As expected, the reliability (SMC) of item parcels is 
relatively higher than that of the individual items. 

To examine the effect of the different numbers of indicators 
per subscale on its reliability (internal consistency), Cronbach's 
a was calculated for each of the four subscales of test anxiety 



* Indicator refers to items or parcels. 
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(see Table 3) . These result shows that the reliabilities did not 
drop when the number of indicators per subscale decreased (e.g., 

.80 in Model A Worry and .80 in Model C VJorry) . 

Internal consistency (Cronbach's a) of each parcel was also 

examined in order to determine whether the item combinations used 
in this study were appropriate. When the number of items per 
parcel is greater than two (Models C, H, and K) , Cronbach's alpha 
for each of the parcels is relatively high (.57 to .79) , thus 
reflecting the internal consistency of the items in each parcel. 

The results of CFA in the first sample are summarized in 
Table 3. When the same niomber of items was included in the model, 
model fit improved as a function of the number of items per 
parcel. GFI, CFI, TLI, and ECVI were improved when the niomber of 
items per parcel was increased. However, the improvement in the 
chi-square to degrees of freedom ratio is not systematic. When 40 
and 36 items were analyzed, the smallest chi-square to degrees of 
freedom ratio corresponded to the models with individual items 
(Model A and Model D) , while the models with the largest number of 
items per parcel yielded the largest chi-square to degrees of 
freedom ratios. However, when 32 and 24 items were analyzed, the 
smallest chi-square to degrees of freedom ratios were found in the 
models with the largest niomber of items per parcel (Model H and 
Model K) , while the models with two items per parcel yielded the 
largest chi-square to degrees of freedom ratios (Model G and Model 
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Replication Sample 

The tendency toward the improvement in normality and 
indicator reliability (SMC) when item parcels are used is similar 
to the one found in the first sample. However, the item level 
normality in the replication sample is somewhat better than its 
counterpart in the first sample (see Table 4 and 5) . 

The results of CFA for the replication sample are summarized 
in Table 6. Like the results in the first sample, GFI , CFI, TLI, 
and ECVI were improved when the number of items per parcel was 
increased. However, the changing pattern of chi-square to degrees 
of freedom ratio was not the same as the one found in the first 
sample. For example, in the first sample, the smallest chi-square 
to degrees of freedom ratio corresponds to model H (4 items per 
parcel, 2 parcels per factor) and model K (3 items per parcel, 2 
parcels per factor) , while the smallest chi-square to degrees of 
freedom ratio in the replication sample corresponds to model C (5 
items per parcel and 2 parcels per factor) . 

Discussion and Cone lus ions 

As expected, smaller skewness and kurtosis and higher 
indicator reliability (SMC) were found in item parcels than those 
in individual items. Also parcels with more items tend to have 
higher reliability than those with fewer items. Improvement in 
normality leads to improvement in reliability because one of the 
assumptions underlying reliability estimation is that scores of 
variables should be normally distributed. 

The reliability (Cronbach's a) of the subscales did not drop 

when the number of the indicators per subscale decreased because 
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the information included in the subscale remains the same. Even 
though the number of indicators decreases, each of these 
indicators includes more variability as a result of extending the 
score range when more items are grouped in one parcel. 

GFI, CFI, TLI, and ECVI improved systeiratically when the 
number of items per parcel increased. This improvement can be 
attributed to several factors. One factor is the decrease in the 
number of parameters to be estimated in each model . When the 
number of parameters to be estimated is small, the probability of 
making specification errors decreases. Hence, the standard error 
of estimation will be small. Improvements in continuity, 
normality, and indicator reliability may also be factors that 
contribute to the better fit of the model . 

In sum, based on the limited design of the current study, it 
is safe to conclude that item parcels function better than 
individual items do when the maximum likelihood estimation method 
is used in CFA. The positive impact of using parcels on model fit 
is more obvious when some of the individual items depart from 
univariate normality and/or have low item reliability. When items 
are similar in content, parcels that consist of more items lead to 
better fit . 

If researchers have a reason to believe that the set of items 
and the specified model represent the construct (s) of interest, it 
is recommended that they not delete items nor change model 
specification based on statistical criteria such as the 
modification index. An alternative way to improve model fit when 
ordered categorical items and maximum likelihood estimation are 
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used is to form parcels based on content similarity. It is more 
likely that the results from this method are replicable because 
this method is not data driven. 

In the current study, the impact of using item parcels is 
limited to one personality measure. Therefore, further studies 
with other measures are needed to support the results of the 
present study. In addition, studies with simulated data are needed 
to confirm the results obtained from the empirical data used in 
this study. 
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Table 1 

Descriptives of Individ ual Items (Sample I). 



Item 


Mean 


SD 


Skew- 

ness 


Kur- 

tosis 


Item Reliabilities (SMCl 
40 36 32 24 


1 


2.26 


0.88 


0.51 


-0.28 


.43 


.43 


.44 




2 


2.19 


1.07 


0.47 


-1.02 


.21 


.22 


.22 


.20 


3 


1.45 


0.80 


1.83 


2.60 


.30 


.31 


.30 


.33 


4 


1.61 


0.87 


1.38 


1.06 


.29 


.28 


.27 


.25 


5 


1.73 


0.89 


1.08 


0.33 




.18 






6 


2.13 


1.08 


0.53 


-1 . 00 


.34 


.35 


.35 




7 


1.55 


0.83 


1.61 


1.98 


.34 


.37 


.38 


.42 


8 


2 .76 


1.02 


-0.17 


-1.16 


.30 


.30 


.31 


.30 


9 


1.92 


1.04 


0.84 


-0.53 




.17 


.18 




10 


1.26 


0.68 


2.69 


$.4_a 


.06 








11 


2.62 


1.06 


0.03 


-1.27 


.40 


.41 


.41 




12 


1.79 


0.88 


0.91 


-0 . 01 


.30 


.31 


.32 


.35 


13 


2.54 


1.08 


-0.02 


-1.28 










14 


1.45 


0.81 


1.83 


2.57 


.23 


.22 






15 


2.38 


0.98 


0.38 


-0.85 


.47 


.48 


. 48 


.47 


16 


2.53 


1.03 


0.14 


-1.17 


.43 


.43 


.43 


.43 


17 


1.60 


0.90 


1.39 


0.87 


.21 


.21 


.23 


.24 


18 


1.33 


0.71 


2.35 


5.07 


.43 


.43 


.39 


.40 


19 


1.85 


0.95 


0.95 


-0 . 05 


.44 


.45 


.45 


.49 


20 


2.58 


1.02 


0.05 


-1 . 14 


.38 


.39 


.41 


.43 


21 


2.40 


0.91 


0.30 


-0.71 


.44 


.44 


.44 


.42 


22 


2.14 


1.08 


0.52 


-1.01 


.27 


.27 


.27 


.27 
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Table 2 



Descriptive Statistics of Item Parcels (Sample 1) 
Model C (5 items per parcel, 2 parcels per subscale) 
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Model H 


(4 items per 


parcel , 


2 parcels per 


subscale) 
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(3 items per 


parcel , 


2 parcels per subscale) 
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Table 3 

Model Co mparison (Sample .I.). 
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Table 4 



Descriptives of Individ ual Items (Sample 2J 



Item 


Mean 


SD 


Skew- 

ness 


Kur- 

tosis 


Item Reliabilities (SMC) 
40 36 32 24 


1 


2.51 


0.80 


0.55 


-0. 


47 


.43 


.44 


.44 




2 


2.31 
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0.22 


-0. 


99 


.28 
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.28 
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1.23 
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52 


.39 


.41 


.43 
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Table 5 



Descriptive Statistics of Item Parcels (Sample 2..). 
Model C (5 items per parcel, 2 parcels per subscale) 













Parcel 


Internal 


Parcel 


Mean 


SD 


Slcew- 


Kur- 


Reliability 


Consistency 








ness 


tosis 


(SMC) 


(Cronbach a) 


W1 


12.50 


3.34 


0.23 


-0.80 


.80 


.74 


W2 


12.84 


3.09 


0.04 


-0.75 


.64 
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TENl 
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-0.06 


.74 


.69 



Model H 


: (4 items per 


parcel , 


2 parcels per 


subscale) 












Parcel 


Internal 


Parcel 


Mean 


SD 


S)cew- 


Kur- 


Reliability Consistency 








ness 


tosis 


(SMC) 


(Cronbach a) 


W1 


9.75 


2.70 


0.25 


-0.65 


.72 


.69 


W2 


10.13 


2.73 


0.12 


-0.70 


.73 


.66 
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Model K 


(3 items per 


parcel. 


2 parcels per 


subscale) 


Parcel 


Mean 


SD 


S)cew- 

ness 


Kur- 

tosis 


Parcel Internal 

Reliability Consistency 
( SMC ) ( Cronbach a ) 


W1 


7.68 


2.12 


0.14 
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.65 


.66 
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2.26 
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.60 


.67 
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.88 


.62 


BSl 
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Table 6 

Model Comparison (Sample 2) 
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Model I 



Each indicator 
is an item 




TTTTTTTTTTTTTTT t TTT 1 1 1 1 




Model K 



Each indicator 
(parcei) 
consists of I 
three item 




Figure 1 . Example of three models with different 
numbers of parcels per factor and different numbers of 
items per parcel . 
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Figure 2. Description of the •'esearch design 

Note. Each model consists of four factors 



